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ABSTRACT ' f 

Edits that can be performed by M;he processor who 
applies for federal campus-based student aid programs are discussed, 
along with a longer-term approach to assessing the efficacy of 
recommended edit checks. Xttention is focused on the accuracy of data 
submitted by institutions on the application portion of the Fiscal 
Operations Report and Application to Participate ( F I SAP ) . Recommended* 
are additional internal edits, as well as cross-year edit checks and 
cross-data source edit checks using Pell Grant and Higher Education "* 
General Information 'Survey data. Recommendations for long-term 
assessment include developing a data gathering and analysis plan for 
analyzing the validity of potential edits and reasonable tolerances 
for* these edits. Principles embodied in the approach include 
comprehensiveness, efficiency, congruity, and verif lability. There is 
a need to clearly define what constitutes an error and to analyze 
data collected at colleges, to determine the comparability of certain 
types of data. The 13 current edit, checks used on the FISAP's 
application portion should be considered for incorporation in the 
FISAP application editing system." (SW) - , 
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'I . , 

INTRODUCTION 

Advanced Technology has conducted, various analyses "associated with the Fiscal 
Operations Report "and AppUcati'ort.to Participate (FISAP) In the Campus-Based 
student aid programs. These analyses are part of the Title IV Quality^Controi^tudy 
that will assess for the-DWision of 'Quality ^ssurance (DQA}, Office of Student 
Financial Assistance^COSFA) the quality of delivery of Campus-Based student aid funds- 
to students enrolled at institutions participating in the three Campus-Based programs. 

' The first activity of the FISAP analysis! involved data collection at the Depart- 
ment of Education (ED). Advanced^jfcchnoiogy data collectors conducted a series of 
manual edits for the 275 institutions participating in the Campus-Based programs that 
were sampled for the Spring 1984 Title IV Quality Control data collection. 

These edits included cross-year comparisons of FISAP data eierrients and cross- 
data source comparisons with the Pell. Grant Disbursment System file and Higher 
Education General Information. Survey (HEG1S) files. Institutions that were Sound to 
have discrepancies in FISAP data elements were identified for further data retrieval. 
The results of this first activity are contained in a_ report entitled "Analysis of Error 
Associated -with the Application and Allocation Aspects of the Campus-Based Pro- 
grams: Results from Initial Collection," submitted to DQA In May 1984. 

The second activity consisted of a supplemental FISAP. data collection conducted 
during, the Title IV data collection at those institutions identified through the manual 
edits described above. Financial aid administrators and registrars were asked to 
resolve extant discrepancies. These data are still being analyzed and a report will be 
produced in the next several months. * 

s • « 

The focus of this report is &e recommendation of edits that can be performed on 
the application portion of the FISAP by the processor including recommended initial 
tolerances for these edit checks. However, since these tolerances have been 
developed through qualitative analytic methods and since data are currently unavail- 
able with which to -establish tolerances through quantative methods, the report also 
proposes a longer-term approach to assessing the- efficacy of recommended edit 
checks, the adequacy of recommended tolerances. This approach also includes 
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'developing a 'data gathering and^anaiysis plan for analyzing the validity of potential 
edits and' reasonable toleranceif or these edits. 



Background and Nature of the Problem 

The application and funding process "for the three . Campus-pased programs, 
College Work-Study (CW-S), National Direct Student Loan (NDSL), and Supplemental 
Educational' Opportunity Grants (SEOG) is complex, time-consuming, and potentially 
e.rror prone. Campus-Sased student financial aid funds are allocated to participating 
institutions in each state. by a process in which the overall financial need of students 
at one eligible institution is compared to the need of .students attending other 
applicant 'institutions in that state and. in the nation as a whole. For both the NDSL 
and CW-S programs, an institution receives an allocation "computed in three general 
stages from data filed by institutions on the application portion of the FISAP: 

(1) " A conditional guarantee 

(2) A state increase based on its "fair share" of the state apportionment 

(3) a national increase based on its "fair share" of the national appropriation. 

if 

m 

f * ' ■ f 

For SEOG, the allocation is • computed in four general stages ttor previous program 
participants): , * I 

(1) A conditional guarantee 

(2) An initial year (IY) state increase based on its "fair share" of the state's IY 
^ a pportionment 

(3) An IY national increase based on its "fair share" of the national IY portion 
* of the SEOG appropriation 

(^) A continuing year (CY) national increase based on -its "fair share" of the 
national CY portion of the SEOG.appropriation. 

Through more than a decade of evolution the attempts to increase the validity of 
application forms and ensure more accurate and more equitable distribution of funds 
have complicated the application as well as application processing. The potential for 
errors by the institutions and the processing system has also increased. Jhis increase 
in complexity and resultant increases in the opportunity .for error, combined with an 
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increasing number of participating institutions (currently over 4,000), have caused a 
good deal of attention to be focused on FISAP processing by OSFA and others. 

This report focuses oh one area that continues to receive attention, namely, the 
( accuracy of data submitted by institutions on the application portion of the FISAP. 

T 

These data are critical since they determine, ' to a degree, the institutions' funding 
level for each of the Campus-Based programs. 

^Application Processing 

A major area of OSFA concern relates to the computerized processing of the 
FISAP by the contractor which produces' an institution's initial allocation. The 
allocation is based -on the funding formula, current regulations, and the program 
appropriations. Last-minute changes to regulations and delays in receiving the 
appropriations result in last-minute modifications to program moduies'mN^ie process- 
ing system. Edit Checks must be modified to accomodate these changes, potentially 
resulting in processing delays. While such modifications^ cannot be controlled, they 
must be considered in developing an approach to systematic application edit checks. 

* \ * 

Ideally, institutions would like to know in s January how much money will be 
available to award to students for the following academic-year. Jn recent years many 
institutions have not received a final allocation before IMfeding aid to students. In' 
part this delay has. been £auseUby the last-minute nature^ffrhe changes noted above 
1 and delays in the funding process. However, when considering a comprehensive system 
of edit checks, care must be taken in order not to overburden a large number of 
institutions with error messages and correction requests which would slow corrections 
and processing required to produce the final allocation. Such edits can quickly reaqh 
the point of diminishing return. 

Follow-up Procedures . 

* • * 

Once schools have received error messages noting specific line items on the 
FISAP which failed to pass system edit checks, it is their responsibility to make the 
necessary corrections and forward .the corrected items to 05FA for reprocessing. 
Although certain uncorrected items will trigger a flag in the system and the institution 
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. will be put on*"hoid," other correction failures go through the system and still permit 
an institution to receive an initial award. In some cases that initial allocation is in 
error and the institution unknowingly awards funds it is not entitled to award. 
Inability of OSFA to follow , up the status of corrections and » appeals to initial 
allocations is 'due in large parKto insufficient stalf to track corrections and appeals 
which annually come from an increasing percentage of over 4,000 schools participating 
in the program. System tracking procedures for* corrections submitted for reprocess- 
ing (e.g., aging reports) and prompt resolution of aiio^l^tions for "holfr' institutions are 
support follow-up procedures which are .contractor responsibilities. Each of these 
areas affects the quality and integrity of the' FISAP process and is another 'aspect of 
OSFA concern. Thjs report deals in a limited manner -wTth these areas.' t 

GENERAL APPROACH TO DEVELOP&G EDIT CHECKS 

The Campus-Based- processor's draft quality control plan identifies 109 edit 
checks for the FISAP. However, only 13 of these edits relate to the application 
portion of, the FISAP./ In addition, these edits are restricted primarily to internal 
arithmetic and consistency tests. ~ 

0 i 

The edit checks recommended in this report include several additional internal 
edits. Also recommended are cross-year edits checks and cross-data source edits 
checks using Pell and HEGIS data. 

m * 

Fundamental Design Principles 

r 

T The^ development • of the series of edit qhecks enumerated in the" following 
sections is based on. several principles which focus on the inherent qualities of an 
effective quality - control system for the application portion of FISAP and its 
relationship to the Campus-Based delivery system, particularly the capabilities of the 
main actors, institutions, and ED. The principles embodied in our approach 
'ude: 

, • Comprehensiveness * 

• Efficiency 

• ^ Congruit/ .* * ~ 
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Verif lability 



Comprehensiveness as a design principle ensures that all meaningful edits will be 
conducted and all possible data sources will be utilized.^ * Efficiency has more 
dimensions as a design principle. The first dimension considers the capability of the 
system to discriminate between erroneous data and reasonable year to year changes in 
data that often occur at institutions subject to dynamic circumstances. A. system of 
edit checks must be^bie to identify large rtes^bers oi errors in a systematic rather 
than random fashion and to target error prone cas£s> 




The second dimension of the efficiency principle is the araity to elicit changes 
in data elements from institutions, and to track and incorporate these changes into the 
processing system expeditiously. Consequently, neither an inordinately large number 
of edits nor an unnecessarily large number of cases, as discussed ^bove, should be 
produced. Either condition would have adverse impacts on institutions and the ED 

staff and pro^essp^ - * • 

* * ** 

The next design principle is congruity, which implies ttW the treatment assigned 
relates directly to the importance of the data item* and the reliability of the data 
sources and tolerance levels used in the edits. The last design principle is to maximize 
the number of edits that have directly verifiable information. 

» . ■-■ ■ ^ . * 

Approach V * 

. / , • 

A two staged approach to^deveioping an effective set of edit checks is proposed. 
This approach is proposed for .two reasons. First, as oufvprior reports indicated, there 
is rto dear definition or determination, of what constitutes an error. Second, a 
preliminary analysis of our data collection at institutions raised significant questions 
about direct comparability c£ certain types of comparison data. This w&uld suggest 
that mofe detailed analysis is required. 

' " The first stage consists, of a' thorough review of the current edits and the 
application document in order to develop, evaluate, and > -recom mend edit ^hecks for 
implementation for this processing year. Potential ^edits were also* identif ied\but are 
not recommended f$r implementation this year. Rather ;h^y are recommended for 

further -anaiy sis and implementation during thepext processing year. Initial tolerance 
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^levels are recommended for this processing year. These ~we/e developed through 
qualitative analysis, including analysis of edit tolerance structures and levels used in 
other, processing systems. First stage activities included the following: 

• Review processor's draft QC plan ' 

• ■ ' ' \ ' " .-.*'*' 

• ' Analyze other processing systems 

• Analyze* proposed, edits and the application document 

• v Review edits proposed in the QC plan, as well as omissions 



Develop 'comprehensive list of potential edits 



• • Identif y data sources^-^ 

• Develop initial tolerances 

• Identify edits to recommend for furthezvanaiysis and possible implementa- 
tion during 198^-85 processing year. / 

x ' ' ' , * 

The results of this phase are presented in the next section. Briefly, the results" 
include additional internal edits not covered in the QC plan; internal or external data 
sources for each of the Ik edits developed; and a tolerance structure that is sensitive 
both to magnitude of the change in data elements and the reiiabili^of the data 
sources used for comparison. 

. . . j 

' The second phase of the proposed approach has two objectives and achieves these 

through developing* an analysis plan and conducting longer te$m* analysis. The" first bl 

these objectives is .to refine the tolerance levels based on current processing year 

performance and cross year analysis. Measures of performance could include the rate 

of ,edit failure, magnitudes and frequencies of corrections. 

" The second objective concerns the cpiiect&n and analysis of data w\th which to 
. assess thef feasibility of edits proposed in the following section but not recommended 

for' implementation during this processing year. In addition, ^analysis will be conducted 

to identify .additional edits or -more elegant and efficient means of conducting edtys. 

The approaches that will be investigated include error prone selection and multi-level 
t edits. .Multi-level edits could include sequencing edjits^o that. an application that fails 

an initial edit would be subject to a battery/of other^ edits to attempt to determine 
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whether the -data are the -result of legitimate changes in characteristics or actual 
errors. ' The feasibility of multi-level treatment for edit failures must also receive 
careful attention. For instance, small changes, even though they are suspected to be 
in error can simply be . flagged and process may continue. However, errors of large 
magnitudes or patterns may require putting an application on hold. * 

We propose to submit such an analysis plan in 60 to SiQ days. 

DESCRIPTION OF POTENTIAL FISAP EDITS * . 

■ * 

The optimum .machine editing* system for processing the application portion of 
the FISAP should perform three functions. The system first must ensure that data 
reported on the FISAP is internally consistent.. Checks should be made to" make 
certain .that the respondent's arithmetic is correct, that, all required entries have been 
made*, and that the relationships between data are logical. Second, since institutions^ 
annually submit the FISAP, the editing system should ensure the cross-year consist- 
ency of data. Most data elements on the FISAP should not change dramatically from 
year-to-year. Inasmuch as the prior year FISAP files can be considered "clean," they 
should be considered as a good base for evaluating .value's on the current year 1 FISAP. 
Finally, the optimum editing system should ensure the consistency of .FISAP data with 
other higher education data collected by the Department of Education. The Pell Grant 
^Disbursement System and HEGIS, foi instance, collect some data which are definition- 
ally consistent with data collected on the FISAP. / . 

Figure 1 shows the 13. edit checks currently' used on the application portion of 
the FISAP. As the figure shows, the .current edits address only the internal 
consistency -of FISAP data. As* such, the current editing system is limited in the kind 
of errors and inconsistencies it can detect. A FISAP form, for example, may have 
data which is. infernally consistent—and, therefore, judged "clean" by the current 
systems-yet have errors which have a direct and possibly significant impact on the 
distribution of Cam pus- Based aid. 

\ ". ' ■ 

Experience^ conducting manual validation of FISAPs from 275 sampled institu- 
tions has suggested 2<Aspecif id edit checks that should be considered for incorporation 
in the FISAP application editing system.* The 2'4 edits which are specified in Figure 2 




BEST COPY AVAILABLE 



ERIC 



. 'f 

1. Institutions with expended funds in the fiscal operations report year, must ' 
designate a request for funds for the upcoming award year on page-i 1, section A x 
lines 1, 2, 3, 4, or 5. • ' 

2. Total ■ expenditures on page 11, line 13 must equal the sum of the individual 
, expenditures in lines 10, 11, and 12. 

■ % - ••• * * 

3. The NDSL FCC request, page 11, section A, line 2 must be less than the NDSL ' 
LOE request, page 11, section A, line 1, 

k. When tuition revenue is on page 13, section E, line 15, column a, enrollment data 
must be entered on section G, line 36, column a,K>r lines 39 through 51, column, 
a. • „ 

5. When tuition revenue is on page 13, section E, line 15, column b, enrollment data 
must be entered on section G, line 37, column b, or lines 39 through 51, column 

6. Total dependent undergraduate eligible aid applicants on page 13, section F, line 
35, column a, must equal the sum of lines 19 through" 34, column a. 

7. Total dependent graduate/professional eligible aid applicants on page 13, section* 
F, line 35, column b must equal the*sum of lines 19 through 34,, column b. 

8. Total independent undergraduate eligible aid' applications on page 13, section F, 
line 35, column c, must equal the sum hi lines 19 through 34, column c. 

9. Total independent graduate/professional eligible aid applicants on page 13, 
section F, line 35, column d must equal thfsum-of lines 19 through 34, ctoiumn d. 

10. " When total number of students is entered on page 13, section G, line 36, columns 

a or b, there must be no entries in lines 39 through 51, columns a and b. 
# 

11. Whin page 13, section G, has entries shown in lines 39 through 51, there must be 
no entries 4how£ in section F, lines 19 through 35, columns b and d, and in 
section G, line 36, column- b« 

12. Total non-traditional continuing students on page 13, section G, line 51, column a 
must equal the sum of lines 39 through 50, column a. <; 

13. Total non-traditional new starts , on page 13, section G, line 51, column b must 4 
equal the sum of lines 39 thVough 50, column b. 

* 

X 

FIGURE 1 

DESCRIPTION OF CURRENT EDh" CHECKS 
\ ON THE APPLICATION PORTION 

* \ - OF THE FISAP 

J 



A. INTERNAL CONSISTENCY CHECKS 

n Data Elements 

• peli Grant Expenditures 

• Undergraduate Dependent Applicants 

• Undergraduate Independent Applicant^ 

• Undergraduate Dependent Applicants 

» 

t Undergraduate Independent Applicants 

• Undergraduate Enrollment — Traditional 

• Continuing Enrollment — Nontraditionai 

• New Enrollment ~ Nontraditionai 

• Graduate Dependent Applicants 

• Graduate Independent Applicants 

• Graduate Enrollment 
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Section and 
Line Number 
. References 

, E16 
* F35 
F35 

F35 

F35 
G36a 
G5la 
C,5ib 



Test/Error Condition 

*< 
- k 

If Pell Grant expenditures were reported in El 6 then 
data for dependent or Independent undergraduates must 
be reported in F35. 



The sum of dependent and independent undergraduates 
reported in F35 must be less than or equal to the under- 
graduate enrollment reported in G36a or the sum o Con- 
tinuing and new enrollment reported in G51a and G$ib. 



F35 
'F35. 
G36b 



The sunvof dependent arid independent graduates/profes- 
sionals reported in £35 must be less than or equal to 
the graduate/professional enrollment reported in G 36b. 



FIGURE 2 * . 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 

CAMPUS-BASED ^(STEM \2 



4 



CROSS-YEAR CONSISTENCY. CHECKS 




B. 



. # , DataJEi 

Institutional Grant Expenditures-! or 
A Award Year 1977-78 « ' *■ - 

. „ , Undergraduate Tuition and Fees 



Graduate Tut 




d Fees 



Section and 
Line Number 
References 

'" E18 * 



El 5a 



E15b 
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Test/Error Condition 



Institutional grant expenditures must equal correspond- 
ing data from last year's fonru ' * 

Current year total undergraduate tuition-and fees wi|l 
be compared with corresponding data f nam last year's * 
form. The current year data must be within the*follow- * 
ing tolerances which vary according to the level of last 
yearVdata: . " ' s 

S 1 ■ ' ft 



Level of Last 
Year's Data 

■ # 

$ - 0 - $ 499,999 
-500,000 - 999,999 
1,000,000 - ,9,999,999 
10,000,000 * 19,99>,999 
20, 000 ,-000 and above 



, Tolerance Ranges r 
for This Year's Data 

_ « . 

+ 60% of last year's data 
+ 50% of last year'sdato 
7 40% of last yeasts data 
+ 30% of last year's data 
+ 20% of last year's data 



Current year total graduate tuition and fees when com- 
pared with corresponding data from last year's form 
must fall withiq the following* tolerance ranges: 



Level of Last 
* ' Year's data 

$ - 0 - $ 4 99, 999 
500/000 - 999,999 
1,000,000 - 9,999,999 
10,000,000 r 19,999,999 
20,000,000 and above 




ance Ranges 
for Tliis Year's Data 

+ 60% of last year's data 
7 50% of last year's data 
+ 40% of last year's data 
7 30% of last year's data 
7 20% of last year's data 
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FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
CAMPUS-BASED SYSTEM 
(Continued) 
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B. CROSS-YEAR CONSISTENCY CHECKS (Continued) 



Data Elgin 



Pell Grant Expenditures 



em$ 



Section and 
Line Number 
. References 

* 

E16 - 



■I 
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■ Test/Error Condition- 

Current year Peil Grant expenditures when compared 
with corresponding data from last -year's form must fall 
within the following tolerance ranges: V 



$ . o 

100,000 
500,000 
1,000,Q00 
5,000,000 



Level of Last 
Year's Data 

- $ 99,999 
499,999 
999,999 

- .4,-999,999 
and above 



i * Tolerance Ranges 
for This Year!* Data 

+ 60% of last year's dala 
7 50% of last year's data 
+ 40% of last year's data 
7 30% of last year's dala 
7 20% of last year's data 



State Grant Expenditures 



E17 



Current year state grant expenditures when compared 
with corresponding data from last year's form must fall 
within the following tolerance ranges: 



Level of Last 
i Year's Data 



$ 0 
100,000 
500,000 
1,000,000 
5,000,000 



- $ % 99,999 

*99,999 
999,999 

- 4,999,999 
and above 



Tolerance Ranges 

for This Year's Data 

* ..• *•' 

+ 60% of last yea?*S$al£* 
7 50% of. last year's dala 
7 40% of last year's data 
7 30% of last year's dala 
*7 20% of last year's data 



FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
t g CAMPUS-BASED SYSTEM 



(Continued) 4 
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B. CROSS-YEAR CONSISTENCY CHECKS (Cootinued) 



6. 



Data Elements 

4 * 

* 4 Undergraduate Dependent Applicants 
• Undergraduate Independent Applicants 



Graduate Dependent Applicants 

t 

Graduate Independent Applicants 



Section and 
Line Number 
References 

' -F35 

F35 



4 

F35 
F35 



1 

V 

I \ 

1 
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Test/Error Condition 



The sum of dependent and independent undergraduates for 
Ihe* current year when Compared with corresponding data 
' from last year's "form must fall within the following 
tolerance ranges: . ' 



' Level of Last 
Year's taata 

f 0: - * ' 99 
100" - ^99 
500 - * 999 
1,000 - H, 999, 

5,000 and above 



Tolerance Ranges 
for This Year's IjJSta 

,+ 60% of last year's data 
7 50% of la'st year's -data 
£ 40% of last year's dafta 
+ 30% of last year's data 
7 20% of last year's data 



(. The sum of dependent and independent graduates for, the 
current year when compared with corresponding data 
from last year's form must fall within.the following tol- 
erance ranges: ' 



.Level of Last 
Year's Data 



0 - 
100 - 
500 - 

1,000 

5,000 



a^d i 



99 
fc99 
999 
4,999 
above 
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FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
CAMPUS-BASED SYSTEM 
(Continued) 



Tolerance Ranges* 
for This Year's Data 

+ 60% of last year's data 
+ 50% of last year's data"? 
7 *f0% of last year's data' 
4. 30% of last yearns data 
7 20% of last year's data 
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B. ' CROSS-YEAR CONSISTENCY'CHECKS (Continued) 
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Data Elements 



8, Undergraduate Enrollment — Traditional 



Section and 
Line Number 
References 

G36a 



V 



Test/Error Condition 



Current year Undergraduate enrollment wfien compared 
»yvith corresponding data from last year's form must fail 
within the following tolerance ranges: 



Level of Last 
Year's Data 



0 - 

100 - 

500 - 

1,000 - 



5,000 and above 



99 J 

999 
4,99£ 



Tolerance Ranges 
for This Year's Data 

* 60% of last year's data 
7 5Q% of last year's data 
+ <f0% of. last year's data 
7 30% of last year's data 
+ 20% df last year's data 



9. Graduate Enrollment 



G36b « 



U> 



Current year graduate/professional enrollment when 
compared with corresponding data from last year's must 
fall within the following tolerance ranges; 



Level of Last 
Year's Data 

0 - 99 

100 - 499 

500 - 999 

1,000 - 4,999 

5,000 and above 



Tolerance Ranges 
for This Year's Data 

4- '60% of last year's data 
+ 50% of last year's data 
+ 40% of last year's data 
7 3fi% of last year's data 
+ 20% of last year's data 
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FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
CAMPUS-BASED SYSTEM 
(Continued) 

4 . . . 



* 50 
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D. CROS£-YEAR CONSISTENCY CHECKS (Continued) 



* Data Elements 
10j • Continuing Enrollment Nontraditional 
• Now Enrollment --Nontraditional 



Section and 
Line Number. 
References * 

. G51a" 

* ' C51b 
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Test/ Error Condition 



The sum of continuing and new enrollment for the cur- 
rent ^ear whemcompared with corresponding data from 
last year's form must fall within the following tolerance 
ranges: , 



Level of Last 
Year's Data 

0 - ; 99 

100 - 499 

500 - ' 999 

1,000 ~~ ~ 4,999 

5,000 and above 



Tolerance Ranges 
for Tliis Year's Data* 

+ 60% of last year's* data 
»T 50% of last year's data 
40% of last year's data 
+ 30% of test year's data 
+ 20% of last year's data 



£ 11. • Total Undergraduate Tuition and Fees 

• Undergraduate Enrollment Traditional 



Continuing Enrollment ~ Nontraditional 
New Enrollment -- Nontraditional 



' 12. r Total Graduate Tuifton and Fees 
t Graduate Enrollment 



El 5a 
G36a 
G5ia 
G5ib 

E15b 
G36b 



If total undergraduate tuition and fees increase by more 
than 10$ when compared with last year's data, undergrad- 
uate enrollment in traditional institutions or the sum ol 
continuing and new enrollment in nontraditional schools 
must not decrease by 10% or more. 



* 



If total graduate tuition and fees increase by more than 
10% when compared with last year's data, graduate en- 
rollment must not decrease by 10% or more. 



9 

ERIC 



21 



FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE liDITS FOR THE 
CAMPUS-BASED SYSTEM 
(Continued) 
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li. CROSS-YEAR CONSISTENCY CHECKS (Continued) 



\ \ 

Data Elements • ' 
1 3. * Total Undergraduate Tuition and Fees^ 
. • Undergraduate Enrollment ~-> Traditional 

• Continuing Enrollment — Nontraditional 

• New Enrollment — Nontraditional 

r 

^ • Total Graduate Tuifc&n and Fees 

• Graduate Enrollment 



15. • P&ll Grant Expenditures 

* i 

• Dependent Undergraduate Applicants 
with Incomes From $0 - $26,999 

• Independent Undergraduate Applicants 
with Incomes From $0 - $8,999 



16.- • Pell Grant Expenditures ' 

v 

• Dependent Undergraduate Applicants 
with Incomes Frcfm $0 - $26,999 

i \ 

• Independent Undergraduate Applicants 
with Income From $0 - $S,999 
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Section and 
Line Number 
References 

EiSa 

G36a 

G51a 

% G>lb 



El5b 
G36b . 

•> 

t 

E16 

* 

F19 through F27 

f 

t 

f F19 through F27 

f V 

E16 
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Test/Error Condition . ■ „ • 

If total undergraduate tuition and fees decrease by 
more than 10% when compared with last year's data, 
undergraduate enrollment in traditional institutions or 
the sum,of continuing and new enrollment in nontradi- 
tional schoolsmost not increase by 10% or more. 



If total graduate tuition and fees decrease. by more than 
10% when compared with last year's data, graduate enroll- 
ment-must not increase by 10% or more. 

If Pell Grant expenditures increase by 10% or more when 
compared with last year'swdata, the sum of dependent 
applicants with incomes below $26,999 and independent 
applicants with incomes below $8,999 must rfot decrease 
by 10% or more. 



F19 though F27 
■ F19 through F27 

* > 

FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
CAMPUS-BASED SYSTEM 
{Continued) 



If Pell Grant expenditures decrease by 10% or more 
when compared with last year's data, the sum of depen- 
dent applicants with incomes below $26,999 and indepen-, 
dent applicants* with incomes below $8,999 must not 
increase by 10% or more. 
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C. CONSISTENCY CHECKS WITH OTHER DAfX BASES 



Data Elements 

1. • Total Undergraduate Tuition and Fees 
- • Total Graduate Tuition and Fees 

/ 

2. Peil Grant Expenditures 



Section and 
Line "Number ' 
fences 

El 5a 

Ei5b 
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Test/Error Condition 

f - 

The sum of tota^tuition and fees for undergraduates and 
graduates must not differ by plus or minus 10% when com- 
pared with comparable data on Par^ A, Line i of HEGIS' j ; 
Financial Statistics Survey. ' »' 

Pell Grant ej^nditqres must^not differ by plus or minus <■ 
10% when compared against comparable data on the Pell; 
Grant Disbursement System's Universe File. 



3. 



Undergraduate Enrollment — Traditional. 
Graduate Enrollment 



G36a 
G36b 



The sum of undergraduate and graduate enrollment must 
not differ by plus or minus 10% wfcm compared- with the 
of lines 1, 10, 11, 15, 2^, and~5 



sum 



. and 14 of HEGlS's Fall Enrollment Survey: 



5 from column^ 1 3 



a. 



• Type of Institution 

• Total Undergraduate Tuition and pees 

• Undergraduate Enrollment 

• Type of Institution 

• Total Graduate Tuition and Fees 

• Graduate Enrollment * 



El 5a 
G36a 

AH 
El 5b 
G36b 



If the institution is private, nonprofit, the ratio of under- 
graduate enrollment to total undergraduate tuition and 
fees must not differ by more than plus or minus 25% when 
compared with undergraduate tuition and fees reported on 
HEGIS' institutional Characteristics Survey. 



If the institution is private, nonprofit, the ratio of grad- 
enrolimeni to total graduate tuition and fees must not 
differ by more than plus or minus 25% when c6mpared , 
with graduate tuition and fees reported on HEGIS' Insti- 
tutional Characteristics Survey. 
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FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
CAMPUS-BASED SYSTEM 
(Continued) 
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address alUtoree :€<dit >che<:k functions* - Additional internal consistency checks, cross- 
.y/ear comparisons, and comparisons with other, higher education data files. Figure 2 . 
'describes each of the proposed edits, provides the data elements which each edit 
addresses, and references the section and line number of tjie data element on the 
form. As Figure 2 indicates, the proposed edits address only those data items in 
Sections E, F, and G on the - application portion of the FISAP which are critical in 
determining an ^jstitution's need for Campus-Based aid. . " ' t 

Proposed Internal Consistency Checks 

Three edits, which would check to make sure that the relationships between 
certain data items are logical are proposed. The first edit would ensure that when Pell 
Grant expenditures are reported in Section E, a figure for total undergraduate eligible 
aid applicants is also reported in Section. F. The second two edits would make sure 
that the number of eligible aid applicants reported in Section F is less than; or equal to 
the institution's total Fail enrollment reported in Section G. (f ' 

Recommendation 

We recommend that OSFA incorporate these three internal consistency checks 
into the FISAP editing system for the upcoming processing year. The three edits 
address Pell Grant expenditures, number of aid applicants, and total enrollment, three 
data elements which our field work suggests are among the most often misreported. 
The three edits, by . checking the consistency of the three data elements, would 
uncover misreporting that the current system is not capable of identifying* 

Proposed Cross-Year Consistenc)^hecks 

Sixteen edits which check an institution's cross-year reporting Consistency are 
proposed. In the first edit, a cross-year comparison is made with not tolerance 
provided. In the next 9 edlts, ( a critical item is compared cross-year, using a tolerance 
range. If the values of the two. items being compared fall within the tolerance range, 
the item would not be considered in error. The proposed tolerance ranges are 
expressed as proportions of current year data to last year's data. .For example, if an 
•institution reported 80 for its undergraduate enrollment last year its tolerance would 
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e plus or minus 60 -percent^ Its tolerance range for this year's data* would 'be 
32 (SO - (.60 x SO) = 32) to 12S (SO + (.60 x = 128). The tolerances vary depending 
•on the amounts reported on last year's form. Ranges are broader for small amounts 
and more restrictive- for large amounts since significantly more cross-year variation 
should be expected at small institutions. 

. ■ • * / * - - 

Extensive quantitative analysis of trends in PlS/KP reporting and the possible 
impact of various tolerance ranges on the frequency of edit failures was not possible 
due to the limited scope of the task.. Instead, the proposed tolerance ranges were 
assigned, based on experience doing manual cross-year comparisons on 275 samples 
FISAPs and on .a review of cross-year edits used by the HEGIS surveys. The HEGIS 
program has been using cross-year comparisons for several years and as a result has 

been able to follow trends and refine the tolerance ranges it uses. 

t . * - ' - • 

The final six cross-year consistency checks examine the relationship of change in 
two or more data elements. In four of the edits, change in total tuition and fees is 
compared to change in total enrollment (e.g., if total tuition and fees increase by more 
than 10 percent, theft enrollment should not decrease by more than 10 percent). In the 
othef-two edits, change in total Pell Grant expenditures is compared to change in the, 
total number of low income eligible aid appikSBYts (e.g., if Pell Grant expenditures 
increase by 10 percent or more, the number of low income applicants should not 
decrease by f& percent or more). A tolerance range of 10 percent has been used in 
these comparisons, since the relationship between the data elements is not perfect. 
For instance, it is possible for total Pell Grant expenditures to rise and the number of- 
low income applicants to drop due to an increase in tuition and fees or a change in the „ 
Family Contribution Schedule. Likewise, enrollment may drop and total tuition and 
fees rise due to an across-the-board increase in tuition* 

Recommendation. < . „ 

. We recommend that OSFA incorporate the 16 cross-year checks on a test basis 
during the upcoming, year recognizing that the proposed tolerance ranges were 
developed through qualitative analytic methods and that additional analysis will be 
needed to refine them. This analysis should be conducted on an ongoing basis in order 
that this refinement continue and the tolerance not become outrcioded. 
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Proposed Consistency Checks With Other Data Bases 

Five edits are proposed which would compare critical' data element^ from the 
FISAP with data collected by HEGIS and the Pell Grant program. Tolerance ranges 
are. recommended for^the comparisons to account for the. following' two problems in 
validating FISAP data with other data sources:., 

• Ongoing, changes to the Pell Universe and HEGIS files. The figure for Pell 
Grant expenditures on the FISAP may not match what the Pell Universe 
File contains for either current authorization -or net expenditures because 
of an onging reconciliation process that continues for months after the 
award year. Also, as part of the editing process the HEGIS files are' 
revised over a period of several months following the original submission of 
the HEGIS form. 

• Definitional Differences. In two of the proposed edits, an average tuition 
and- fees figure would be calculated (total tuition and fees divided by 
enrollment) and compared against a tuition and fees figure reported for 
private institutions on HEGIS' Institutional Characteristics Survey.* That 
survey, however, asks' for the modal (most common) tuition and fees rather 
than the campus-wide average. Thus, a rather liberal tolerance of 25 
-percent is suggested in making the' comparison. - • • 

Recommendation v - 

We recommend that ways of overcoming the problems we identified in the 
manual validation of 275 sampled forms be explored before incorporating comparisons 
with outside daja bases into the current editing system. These problems, in addition to 
the two comparison problems already noted, include: • ' 

• Difficulty of Identifying Institutions. The HEGIS and FISAP files have no 
common identified for the institution other than its name. HEGIS uses the 
FICE code as the numeric identifer for its institutions," while .FISAP uses 

- the entity number. « . 



♦There is not comparable data for total public institution tuition 'and fees on the" 
Institutional Characteristics Survey. : Data is collected on. that survey for in-state ar\d 
out-of-state tuition and fees'" for public institutions. Without data on , in-state and out- 
of-state enrollments, it is impossible to calculate a campus-wide average tuition and 
fees figure. • 
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Different Reporting Units. The definition of a reporting unit is different 
for FISAP, Pell, and HEGIS. A multi-campus institution, for example, may 
tile a single combined report for FISAP and separate, reports for PeiL 

*■ ■ 

Different Number of Reporting Units. Not ail institutions, who file a "FISAP 
file Peii and HEGIS reports. For example, therj are many proprietary 
schools woo file the FISAP form but not the HEGIS surveys. 
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